After completing this lesson, you’ll be able to:
In the classical sense, prototyping means creating an incomplete application to evaluate a project's feasibility.
Here, we'll stretch the definition to mean how to build a complex FME project incrementally, starting with an empty workspace and building it piece by piece to deliver a result that matches the final specification.
We will cover the techniques used for incrementally building a workspace.
The critical development technique for FME workspaces is incremental updates.
The steps to this technique are:
Although a range of 3-10 transformers is an arbitrary number, the more transformers you add, the more difficult it will be to identify the source of any problems. Beyond ten transformers is the point at which you should consider chopping that process into smaller sections.
Here, an author has planned their workspace by laying it out as a set of bookmarks on the canvas:
Now, the author can complete and test each section simultaneously, keeping the overall goal in mind.
It can be easy to start developing a workspace and forget to save it! FME keeps a recovery file as soon as the workspace is saved for the first time, but until then, you risk an irretrievable loss.
When the FME project is large and complex, the source data will likely be large and complex, too. So, when creating a workspace in small increments and testing each part, it's better to avoid using the entire dataset. It's best to create a small sample of data - extracting it from your source and writing to a neutral format like FFS.
Sampling is beneficial for databases because it also avoids the problems of waiting for network traffic and database responses.
Here, the workspace author extracts a sample of source data by reading from a database, splitting it into tiles, and writing just one tile to the FFS format. While prototyping a solution, the author can use this single tile to represent the entire source database table.
Another transformer to use would be the Sampler, although the features selected by it would not be spatially adjacent.
It's easy to work on a single workspace file when making incremental changes to a workspace. However, there are various problems with this:
Therefore, it is better to keep versioned workspaces, where a different copy is kept for each set of revisions. This precaution can be taken manually within the file system or using a version control system like Git.
It is a good idea to keep and version all materials related to an FME project, including:
It's better not to store any personal information, including passwords. Also, you don't need to store temporary files.
You can use a few methods to keep track of version and editing history:
For additional information on Workspace Comparison, please see the Compare Workspaces course.
You can integrate Git into your FME Workbench comparison workflow. You can use the command line, TortoiseGit, or Sourcetree to interface directly with the Compare Workspaces tool.
